Papers with annotation approach
Cross-document coreference: An approach to capturing coreference without context (D19-62)
Copied to clipboard
| Challenge: | a cross-document coreference annotation schema was developed to extract timelines in the clinical domain. |
| Approach: | They propose a cross-document coreference annotation schema that is governed by schematic rules to create meaningful and consistent cross- document relations. |
| Outcome: | The proposed approach produces an agreement score of 93.77% for identical relations between the two sets of notes. |
Design Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Design (2023.tacl-1)
Copied to clipboard
| Challenge: | Disagreement in natural language annotation has been studied from a perspective of biases introduced by the annotators and the annotation frameworks. |
| Approach: | They propose to analyze task design bias in crowdsourced annotations where lay annotators are used to elicit interpretations. |
| Outcome: | The proposed methods can push annotators towards certain relations and some discourse relation senses can be better elicited with one or the other approach. |
A Corpus for Argumentative Writing Support in German (2020.coling-main)
Copied to clipboard
| Challenge: | In today's world most information is readily available. Consequently, the sole reproduction of information is losing attention. |
| Approach: | They propose an annotation approach to capture claims and premises of arguments and their relations in student-written peer reviews on business models in german language. |
| Outcome: | The proposed annotation scheme guides annotators to moderate agreement with the proposed scheme on 50 persuasive student-written peer reviews on business models. |
MS-Mentions: Consistently Annotating Entity Mentions in Materials Science Procedural Text (2021.emnlp-main)
Copied to clipboard
Tim O’Gorman, Zach Jensen, Sheshera Mysore, Kevin Huang, Rubayyat Mahbub, Elsa Olivetti, Andrew McCallum
| Challenge: | Material science synthesis procedures require high-quality annotations, which are limited by the size and quality of the annotations. |
| Approach: | They propose a corpus of entity mention annotations over 595 Material Science synthesis procedures. |
| Outcome: | The proposed approach greatly expands the training data available for the Named Entity Recognition task. |
Appraisal Framework for Clinical Empathy: A Novel Application to Breaking Bad News Conversations (2024.lrec-main)
Copied to clipboard
| Challenge: | Empathy is essential in healthcare communication. |
| Approach: | They propose an annotation approach that draws on well-established frameworks for clinical empathy and breaking bad news conversations for considering the dynamic dynamics of discourse relations. |
| Outcome: | The proposed model can be used to train models to detect causal relations involving empathy, a feature of systems that can provide feedback to medical professionals in training. |
MOSPC: MOS Prediction Based on Pairwise Comparison (2023.acl-short)
Copied to clipboard
| Challenge: | et al., 2016a) show that MOS prediction model can improve ranking accuracy of speech quality. |
| Approach: | They propose a general framework for MOS prediction based on pair comparison . they use C-Mixup algorithm to enhance generalization performance of MOSPC . |
| Outcome: | The proposed model outperforms baselines on most correlation coefficient metrics . it also surpasses the strong baseline in ranking accuracy on each fine-grained segment. |
Structured Persuasive Writing Support in Legal Education: A Model and Tool for German Legal Case Solutions (2023.findings-acl)
Copied to clipboard
| Challenge: | An annotation approach for capturing structured components and arguments in legal case solutions of German students is proposed based on the appraisal style, which dictates the structured way of persuasive writing in law. |
| Approach: | They propose an annotation scheme with annotation guidelines that identify structured writing in legal case solutions. |
| Outcome: | The proposed approach captures the structure of a persuasive legal text and can be predicted using transformer-based models. |
A Virtual Patient Dialogue System Based on Question-Answering on Clinical Records (2024.lrec-main)
Copied to clipboard
| Challenge: | a new approach to annotating medical dialogues with intents is proposed for virtual patients . a VP is a system that allows medical students to simulate a real clinical consultation . |
| Approach: | They propose to annotate medical dialogue questions in Spanish and a second dataset of dialogues using a novel annotation approach. |
| Outcome: | The proposed approach eliminates the need for manually structured patient records . the two datasets and the code will be freely available for the research community. |
Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for training effective PRMs focus on the first incorrect step and all preceding steps, assuming that all subsequent steps are incorrect. |
| Approach: | They propose a data annotation method specifically designed to score the long CoT reasoning process by using an LLM-based judger for annotation. |
| Outcome: | The proposed method improves PRMs' ability to identify effective self-correction behaviors and reasoning based on erroneous steps. |
Abstract Meaning Representation of Constructions: The More We Include, the Better the Representation (L18-1)
Copied to clipboard
Claire Bonial, Bianca Badarau, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Tim O’Gorman, Martha Palmer, Nathan Schneider
| Challenge: | Abstract Meaning Representation (AMR) uses a flexible pattern or template of multiple lexical items to provide semantic representation of certain constructions. |
| Approach: | They propose to expand the AMR project's lexicon of predicate senses to include entries for a growing set of constructions. |
| Outcome: | The proposed approach provides coverage for the annotation of certain types of constructions. |
Supporting Cognitive and Emotional Empathic Writing of Students (2021.acl-long)
Copied to clipboard
| Challenge: | Empathy skills are an elementary skill in society for daily interaction and professional communication and are therefore elementary for educational curricula. |
| Approach: | They propose an annotation approach to capture emotional and cognitive empathy in student-written peer reviews on business models in germany. |
| Outcome: | The proposed annotation scheme guides annotators to a substantial to moderate agreement with the model and shows that it is effective. |
A Semi-Automated Live Interlingual Communication Workflow Featuring Intralingual Respeaking: Evaluation and Benchmarking (2022.lrec-1)
Copied to clipboard
| Challenge: | Traditionally, live interlingual communication has been achieved only with the help of human interpreters. |
| Approach: | They propose a semi-automated workflow which uses a human respeaker and speaker-dependent speech recognition software to deliver punctuated same-language output of superior quality than the out-of-the-box ASR system. |
| Outcome: | The proposed workflow produces a similar quality output to the best-in-class simultaneous interpreters working with the same source speeches from the European Parliament. |
Structured Interpretation of Temporal Relations (L18-1)
Copied to clipboard
| Challenge: | Temporal relations between events and time expressions are often modeled in an unstructured manner, resulting in inconsistent and incomplete annotation and computational modeling. |
| Approach: | They propose an annotation approach where events and time expressions form a dependency tree in which each dependency relation corresponds to an instance of temporal anaphora. |
| Outcome: | The proposed approach annotates 235 documents in news and narratives with 48 doubly annotated documents. |